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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)[§] This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 1-51 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) E3 Claim(s) 1-27 and 29-51 is/are allowed. 

6) D Claim(s) is/are rejected. 

7) S Claim(s) 28 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
Priority under 35 U.S.C. §§119 and 120 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

ajDAII b)D Some*c)n None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

13) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application) 

since a specific reference was included in the first sentence of the specification or in an Application Data Sheet. 
37 CFR 1.78. 

a) □ The translation of the foreign language provisional application has been received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 since a specific 

reference was included in the first sentence of the specification or in an Application Data Sheet. 37 CFR 1.78. 
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DETAILED ACTION 



Allowable Subject Matter 



1 . Claim 28 is objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

2. The following is a statement of reasons for the indication of allowable subject 
matter: Prior Art of record fails to teach or reasonably suggest a plurality of annotations 
having equal time information and a viewing requesting which annotation of the plurality 
is to be displayed. 



3. The information disclosure statement filed 12 December 2001 fails to comply with 
37 CFR 1 .98(a)(2), which requires a legible copy of each U.S. and foreign patent; each 
publication or that portion which caused it to be listed; and all other information or that 
portion which caused it to be listed. It has been placed in the application file, but the 
information referred to therein has not been considered. 



4. Claims 15, 39, and 45 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. 



Information Disclosure Statement 



Claim Rejections - 35 USC §112 
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Art Unit: 2611 

Regarding claims 15 and 45, claims 15 and 45 recite, "said mask data and said 
at least one of textual data and graphics data are transmitted over separate streams in 
said broadcast channel." It is unclearly which (mask, textual, graphics, or video) data 
are transmitted on over a separate stream. For the purposes of examination, the claim 
will receive the broadest reasonable interpretation of any of mask, textual, graphics, or 
video data. 

Regarding claim 39, it is unclear how the location reference can be both the 
upper left most pixel (as recited in claim 38) and a centroid pixel as recited in claim 39. 
For the purposes of examination, the claim 39 will depend from claim 37. 



5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21 (2) 
of such treaty in the English language. 

6. Claims 24, 25, 27, and 29 are rejected under 35 U.S.C. 1 02(e) as being 
anticipated by U.S. Patent 6,615,408 to Kaiser et al. 

Regarding claims 24 and 29, Kaiser teaches a reproduction apparatus (fig. 1 , 
label 1300) in communication with a broadcast channel (col. 4, II. 59-67; col. 5, II. 22- 
31), a display (fig. 1, label 1200) (claimed display device). Kaiser teaches a receiver 



Claim Rejections - 35 USC § 102 
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decoding a digital signal to recover a video signal (such as high definition television 
formats, see col. 5, II. 30) and annotation data (fig. 2). Kaiser teaches displaying 
annotation information in response to a viewer request (fig. 5, col. 9, II. 37-65), which is 
available on a frame-by-frame basis. 

Regarding claim 25, Kaiser teaches placement zones synchronized to the display 
of the video (fig. 2). 

Regarding claim 27, Kaiser clearly synchronizes the placement zones and the 
video in response to timing information in order to place the placement zones over the 
proper location, such as a car as shown in figures 6A-6D. 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

8. Claims 1, 31, 33, 34, 44, and 46 are rejected under 35 U.S.C. 103(a as being 
unpatentable over U.S. Patent 5,774,664 to Hidary et al. 

Regarding claims 1 and 31 , Hidary teaches a content creation device (fig. 1, label 
4) which clearly has a video source and produces a transport stream, which is inherent 
to MPEG-2 (col. 4, II. 36-39). After the video program is created, Hidary teaches an 
annotation source, such as Uniform Resource Locators (URLs) (col. 4, II. 40-47). 
Whereas Hidary teaches inserting the URL into the vertical blanking Interval (VBI) of a 
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signal, Hidary clearly teaches MPEG-2 transmissions, which do not have a VBI. Hidary 
does not explicitly teach a multiplexer. Official Notice is taken that multiplexers are well 
known in the art. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify Hidary by using a multiplexer in order 
to transmit the URL as a data packet within the transport stream thereby efficiently 
sending information within a given channel. The embedded URL contains timing 
information in order to present the URLs at the appropriate time (col. 5, II. 21 - 33, col. 
7, II. 31-33), which reads on synchronizing the annotation data with the video signal. 

Regarding claims 33 and 34, Hidary teaches using annotation data for 
information regarding goods and services (col. 2, II. 33-40, col. 8, II. 18-44) and non- 
commercial information (col. 2, II. 43-48, col. 8, II. 18-44). 

Regarding claim 44, Whereas Hidary teaches inserting the URL into the vertical 
blanking Interval (VBI) of a signal, Hidary clearly teaches MPEG-2 transmissions, which 
do not have a VBI. Hidary does not explicitly teach a multiplexer. Official Notice is 
taken that multiplexers are well known in the art. Therefore, it would have been obvious 
to one of ordinary skill in the art at the time the invention was made to modify Hidary by 
using a multiplexer in order to transmit the URL as a data packet within the transport 
stream thereby efficiently sending information within a given channel. 

Regarding claim 46, Hidary teaches a broadcasting, as shown in figure 1 . 
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9. Claims 2 and 32 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 5,774,664 to Hidary et al. in view of U.S. Patent 6,415,438 to Blackketter et 
al. 

Regarding claims 2 and 32, Hidary teaches inserting the URL at the time it is 
needed; but is silent on the timing information comprising one of a timestamp, timecode, 
frame numbering, or global time of day. Blackketter teaches inserting triggers with a 
time attribute (col. 4, II. 64-67), such as a frame number (col. 6, II. 16-22). Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to modify Hidary by using time information such as a frame number as taught by 
Blackketter in order to eliminate a delay loop (Blackketter: col. 2, II. 59-61 ). 

10. Claims 3-12, 14, 16-18, 20-23, and 35-43 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over U.S. Patent 5,774,664 to Hidary et al. in view of U.S. Patent 
6,615,408 to Kaiser et al. 

Regarding claims 3, 4, 14, 16, 35, 36, Hidary teaches a URL which reads on 
textual data; but is silent on mask data and location information. Kaiser teaches a 
placement zone which references a product being displayed, wherein the placement 
zone is a location of an object (Abstract). Therefore, it would have been obvious to one 
of ordinary skill in the art at the time the invention was made to modify Hidary by using a 
placement zone (claimed mask data) including location information as taught by Kaiser 
in order to increase interactivity and enable the user to gather more information on a 
product. 
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Regarding claims 5-7, 37-39, the combination of Hidary and Kaiser has been 
explained above. Further, Kaiser teaches placement zones for various frames (fig. 2, 
col. 6, II. 9-17), wherein the placement can track an image at the upper left corner (col. 
10, II. 34-38). Whereas Hidary and Kaiser are silent on a location reference at the 
centroid pixel, Official Notice is taken that a center position is well known in the art. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Hidary and Kaiser by using a center position (as a 
reference point) in order to properly track an image thereby increasing the effectiveness 
of the placement zone. 

Regarding claims 8-10, 17, 40-42, the combination of Hidary and Kaiser has 
been explained above. Kaiser teaches location and shape information such as the 
shape and location of the car (col. 10, II. 34-38), see figures 6A-6D, wherein the visual 
highlight (fig. 6B, label 6500) is a graphical overlay and has an outline of the car. 

Regarding claims 1 1 and 43, Hidary and Kaiser are silent on a mathematical 
representation of set of pixels. Official Notice is taken that mathematical representation 
of pixels are well known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Hidary and Kaiser 
by using a mathematical representation in order to save bandwidth and process the 
information at the client side. 

Regarding claims 12 and 18, Hidary teaches a content creation device (fig. 1, 
label 4) which clearly has a video source and produces a transport stream, which is 
inherent to MPEG-2 (col. 4, II. 36-39). After the video program is created, Hidary 
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teaches an annotation source, such as Uniform Resource Locators (URLs) (col. 4, II. 
40-47). Whereas Hidary teaches inserting the URL into the vertical blanking Interval 
(VBI) of a signal, Hidary clearly teaches MPEG-2 transmissions, which do not have a 
VBI. Since there is no other place for the data other than the transport stream, Hidary 
teaches multiplexing the supplemental (URL) data into the transport stream to create an 
augmented transport stream for transmissions (col. 36-39). The embedded URL 
contains timing information in order to present the URLs at the appropriate time (col. 5, 
II. 21 - 33, col. 7, II. 31-33), which reads on synchronizing the annotation data with the 
video signal. Hidary teaches a broadcast channel as shown in figure 1, (video with 
URLs), a local URL decoder (claimed receiver), and display monitor (claimed display 
device). Hidary is silent on displaying the URLs in response to a viewer request. Kaiser 
teaches displaying annotation information in response to a viewer request (fig. 5, col. 9, 
II. 37-65) in order to display more information. Therefore, it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to modify Hidary by 
receiving a viewer request to display annotation information as taught by Kaiser in order 
to display information pertinent to the user. 

Regarding claim 20, Kaiser teaches a back channel (fig. 1, label 1400-data 
network). 

Regarding claim 21 , Hidary and Kaiser are silent on a store and forward channel. 
Official Notice is taken that a store and forward channel is well known in the art. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Hidary and Kaiser by implementing a store and forward 
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channel in order to efficiently manage the bandwidth of the system by only using 
resources when necessary. 

Regarding claims 22 and 23, Hidary teaches using annotation data for 
information regarding goods and services (col. 2, II. 33-40, col. 8, II. 18-44) and non- 
commercial information (col. 2 ? II. 43-48, col. 8, II. 18-44). 

11. Claims 13 and 1 9 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over U.S. Patent 5,774,664 to Hidary et al. and U.S. Patent 6,615,408 to Kaiser et al. in 
view of U.S. Patent 6,415,438 to Blackketter et al. 

Regarding claims 13 and 19, Hidary teaches inserting the URL at the time it is 
needed; but is silent on the timing information comprising one of a timestamp, timecode, 
frame numbering, or global time of day. Blackketter teaches inserting triggers with a 
time attribute (col. 4, II. 64-67), such as a frame number (col. 6, II. 16-22). Therefore, it 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to modify Hidary by using time information such as a frame number as taught by 
Blackketter in order to eliminate a delay loop (Blackketter: col. 2, II. 59-61 ). 

12. Claims 15 and 45 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over U.S. Patent 5,774,664 to Hidary et al. and U.S. Patent 6,615,408 to Kaiser et al. in 
view of U.S. Patent 5,541 ,662 to Adams et al. 

Regarding claims 15 and 45, Hidary teaches a URL which reads on textual data; 
but is silent on mask data and location information. Kaiser teaches a placement zone 
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which references a product being displayed, wherein the placement zone is a location of 
an object (Abstract). Therefore, it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify Hidary by using a placement zone 
(claimed mask data) including location information as taught by Kaiser in order to 
increase interactivity and enable the user to gather more information on a product. 
Hidary and Kaiser are silent on the mask data and one of textual or graphics data 
transmitted over separate streams in the broadcast. Adams teaches mask data and 
textual or graphics data transmitted as data packets in the transport stream, wherein the 
data packet is separate from the audio and video packets. 

13. Claims 26, 30, and 48 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 6,615,408 to Kaiser et al. in view of U.S. Patent 
6,415,438 to Blackketter et al.. 

Regarding claims 26, 30, and 48, Kaiser is silent on the timing information 
comprising one of a timestamp, timecode, frame numbering, or global time of day. 
Blackketter teaches inserting triggers with a time attribute (col. 4, II. 64-67), such as a 
frame number (col. 6, II. 16-22). Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by using 
time information such as a frame number as taught by Blackketter in order to eliminate 
a delay loop (Blackketter: col. 2, II. 59-61). 
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14. Claims 47, 49-51 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over U.S. Patent 6,615,408 to Kaiser et al. 

Regarding claim 47, Kaiser teaches a reproduction apparatus (fig. 1, label 1300) 
in communication with a broadcast channel (col. 4, II. 59-67; col. 5, II. 22-31), a display 
(fig. 1 , label 1200) (claimed display device). Kaiser teaches a receiver decoding a 
digital signal to recover a video signal (such as high definition television formats, see 
col. 5, II. 30) and annotation data (fig. 2). Kaiser teaches displaying annotation 
information in response to a viewer request (fig. 5, col. 9, II. 37-65), which is available 
on a frame-by-frame basis. Kaiser is silent on demultiplexing, however Official Notice is 
taken that demultiplexing a signal is well known in the art. Therefore it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
Kaiser by demultiplexing the signal in order to process the information the further 
decode the data to supply supplemental information. 

Regarding claim 49, Kaiser teaches a back channel (fig. 1, label 1400-data 
network). 

Regarding claim 50, Kaiser teaches product selection or information (col. 3, II. 3- 
5), collecting user information (col. 14, II. 3-21), and accepting information regarding a 
commercial transaction and completing the transaction (fig. 9). 

Regarding claim 51, Kaiser performs a 'secure transaction verification' procedure 
(fig. 9, label 9300), which in the broadest reasonable sense has some information 
regarding a viewer identifier in order to identify the viewer. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Andrew Y Koenig whose telephone number is (703) 
306-0399. The examiner can normally be reached on M-Th (7:30 - 6:30). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Faile can be reached on (703) 305-4380. The fax phone number 
for the organization where this application or proceeding is assigned is (703) 872-9314. 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (703) 305- 
4700. 



ayk 




